Fix running under nightly: copy prebuilt libtensorflow.so to OUT_DIR
#72
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
A change in cargo nightly (merged on February 7th 2017) changed how dynamic libraries are loaded. This prevented the `tensorflow-sys` crate's tests to run. To fix, simply copy the `libtensorflow.so` pre-built library to `OUT_DIR` and change `cargo:rustc-link-search=` path so cargo can find it when running the tests. See: * issue: tensorflow#71 * cargo's pull request: rust-lang/cargo#3651
nbigaouette-eai
force-pushed
the
fix_library_loading_on_nightly
branch
from
a4b01ad
to
b6bf663
Compare
adamcrume
requested changes
Mar 18, 2017
| println!("cargo:rustc-link-search={}", lib_dir.display()); | ||
| let output = PathBuf::from(&get!("OUT_DIR")); | ||
| let new_library_full_path = output.join(&library_file); | ||
| if new_library_full_path.exists() { |
There was a problem hiding this comment.
I don't think deleting the old one is necessary since the copy will overwrite it anyway.
There was a problem hiding this comment.
That what I though, and that's what the documentation mentions. But unwrap()ing the result fails. For example if I comment out the exists() block and simply copy() I get this:
-> cargo test
Compiling tensorflow-sys v0.7.0 (file://${HOME}/tensorflow_rust.git/tensorflow-sys)
error: failed to run custom build command for `tensorflow-sys v0.7.0 (file://${HOME}/tensorflow_rust.git/tensorflow-sys)`
process didn't exit successfully: `${HOME}/tensorflow_rust.git/target/debug/build/tensorflow-sys-28fb5808195bfa7e/build-script-build` (exit code: 101)
--- stdout
libtensorflow-sys/build.rs:80: binary_url = "https://storage.googleapis.com/tensorflow/libtensorflow/libtensorflow-cpu-darwin-x86_64-1.0.0.tar.gz"
libtensorflow-sys/build.rs:84: base_name = "libtensorflow-cpu-darwin-x86_64-1.0.0"
libtensorflow-sys/build.rs:93: file_name = "${HOME}/tensorflow_rust.git/tensorflow-sys/target/libtensorflow-cpu-darwin-x86_64-1.0.0.tar.gz"
libtensorflow-sys/build.rs:209: Executing "ls" "-l" "${HOME}/tensorflow_rust.git/tensorflow-sys/target/libtensorflow-cpu-darwin-x86_64-1.0.0/lib"
total 209520
-r-xr-xr-x 1 nicolas staff 107270400 31 déc 1969 libtensorflow.so
libtensorflow-sys/build.rs:213: Command "ls" "-l" "${HOME}/tensorflow_rust.git/tensorflow-sys/target/libtensorflow-cpu-darwin-x86_64-1.0.0/lib" finished successfully
cargo:rustc-link-lib=dylib=tensorflow
libtensorflow-sys/build.rs:134: Copying ${HOME}/tensorflow_rust.git/tensorflow-sys/target/libtensorflow-cpu-darwin-x86_64-1.0.0/lib/libtensorflow.so to ${HOME}/tensorflow_rust.git/target/debug/build/tensorflow-sys-adb04dcfd9da710b/out/libtensorflow.so...
--- stderr
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { repr: Os { code: 13, message: "Permission denied" } }', /Users/rustbuild/src/rust-buildbot/slave/stable-dist-rustc-mac/build/src/libcore/result.rs:868
note: Run with `RUST_BACKTRACE=1` for a backtrace.
After investigation, it seems the .so file is -r-xr-xr-x (read and execute only), so the copy fails if the file already exists (it can't be overwritten as it's not writable. Making the file writable allow the copy operation to succeed.
So for the copy to succeed, I either have to chmod the .so file or delete the old one... Any preference?
There was a problem hiding this comment.
Huh, I guess that makes sense. Let's keep the delete code. The fewer platform-dependent requirements (like chmod) that we introduce, the better.
tensorflow-sys/build.rs
Outdated
| std::fs::remove_file(&new_library_full_path).unwrap(); | ||
| } | ||
|
|
||
| println!("Copying {} to {}...", library_full_path.display(), new_library_full_path.display()); |
There was a problem hiding this comment.
Please use the log! macro.
There was a problem hiding this comment.
Done in 05d4d2b
adamcrume
approved these changes
Mar 19, 2017
|
Thanks! |
ramon-garcia
pushed a commit
to ramon-garcia/tensorflow-rust
that referenced
this pull request
May 20, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A change in cargo nightly (merged on February 7th 2017) changed how
dynamic libraries are loaded. This prevented the
tensorflow-syscrate'stests to run.
To fix, simply copy the
libtensorflow.sopre-built library toOUT_DIRand change
cargo:rustc-link-search=path so cargo can find it whenrunning the tests.
Note that I believe the library compiled from source does not seems to be
affected by the problem as it is already copied to
OUT_DIR. This wouldneed to be verified though.
See: